1. Field of the Invention
The present invention generally relates to the field of memory management and more specifically to a configurable cache occupancy policy.
2. Description of the Related Art
One element of a memory subsystem within certain processing units is a Level 2 Cache memory (referred to herein as “L2 cache”). The L2 cache is a large on-chip memory that serves as an intermediate point between an external memory (e.g., frame buffer memory) and an internal client of the memory subsystem (referred to herein as the “clients”).
As is well known, a cache “miss” occurs when a client requests data from the L2 cache, but the data is not currently stored in the L2 cache. The requested data must then be retrieved from external memory while the client that made the request idles. Retrieving data from external memory is far more time consuming than retrieving the data from the L2 cache. Having too many cache misses can cause poor application performance by creating a bottleneck at the external memory. After a cache miss, the L2 cache evicts one or more data entries and replaces that old data with the requested data associated with the cache miss. Problems also can arise, though, when the L2 cache evicts data, and another client then requests the evicted data a short period of time later. Since cache misses can reduce application performance, striking a balance between evicting data too infrequently, which leads to cache misses, and evicting data too quickly, which also leads to cache misses, is important, especially with applications that require high data throughput such as graphics applications.
One difficulty in determining which data to evict from a cache is that the operations transmitted from different clients to the cache may not always have uniform attributes. For example, the operations from one client may require smaller, but higher priority, blocks of data, while operations of another client may require larger, but lower priority, blocks of data. In addition, the operations of some clients may reuse data, while the operations of other clients may require new data. These different data type and data use patterns very much complicate the cache policies that should be implemented for a complex cache, like the L2 cache, that stores different types of data for different types of clients.
As the foregoing illustrates, what is needed in the art is an efficient policy for evicting data from a cache in a parallel processing architecture that receives data requests from multiple and different types of clients.